±WS^>  ■■■:■■■  ::/■:'■ 


THE  UNIVERSITY 

OF  ILLINOIS 

LIBRARY 

370 
ie6e 

no.36-47 


fxr^r°-sb^,he 


University 


of  Illinois  Library 


UNIVERSITY  OF  ILLINOIS  BULLETIN 

Issued  Weekly 
Vol.  XXIII  May  11,  1926  No.  36 

[Entered  as  second-class  matter  December  11,  1912,  at  the  post  office  at  Urbana,  Illinois,  under  the 
Act  of  August  24,  1912.  Acceptance  for  mailing  at  the  special  rate  of  postage  provided  for  in 
section    1103,   Act  of  October  3,    1917,   authorized  July   31,    1918.] 


EDUCATIONAL  RESEARCH  CIRCULAR  NO.  44 


BUREAU  OF  EDUCATIONAL  RESEARCH 
COLLEGE  OF  EDUCATION 

OBJECTIVE  MEASUREMENT 
OF  INFORMATION 

By 

Charles  W.  Odell 

Assistant  Director,  Bureau  .of 
Educational  Research 


m  umm  o»  m 

APR  2  u  1926 
mivmw)/  m  ill; 


PUBLISHED  BY  THE  UNIVERSITY  OF  ILLINOIS 
URBANA 


3^0 

OBJECTIVE  MEASUREMENT  OF  INFORMATION 

Recent  discussion  of  objective  tests.  The  so-called  "new  examina- 
tion"1 has  been  one  of  the  most  frequently  discussed  topics  in  educational 
literature  during  the  last  four  or  five  years.  Many  writers  and  speakers 
have  advocated  that  objective  tests2  be  employed  either  in  conjunction 
with  examinations  of  the  traditional  type3  or  in  place  of  such  examina- 
tions. Although  many  of  those  who  have  done  so  have  based  their 
opinions  largely  upon  theoretical  arguments,  others  have  presented 
experimental  data  to  support  their  judgments.  Among  the  latter  are 
Wood4  and  Ruch,5  both  of  whom  strongly  recommend  the  use  of  ob- 
jective exercises.  Another  experimenter,6  however,  has  concluded  that 
under  satisfactory  conditions  there  is  little  difference  in  the  general 
merit  of  the  two  kinds  of  measuring  instruments.  The  bulk  of  opinion, 
however,  seems  to  be  in  favor  of  some  use  of  the  newer  type. 

Purpose  of  this  circular.  It  is  the  purpose  of  the  writer  to  discuss 
very  briefly  the  place  and  merits  of  the  new  examination,  to  follow  this 
by  illustrating  and  explaining  different  forms  of  objective  and  near- 
objective  tests,  and  to  give  directions  for  administering  and  scoring 
them. 

Function  and  limitations  of  objective  tests.  Although  some  en- 
thusiastic advocates  of  objective  tests  have  urged  that  they  replace  dis- 
cussion and  essay  examinations  entirely,  the  writer  does  not  believe 
that  this  viewpoint  is  justified.  They  have  certain  definite  advantages7 
over  traditional  examinations  and,  in  his  opinion,  should  be  used  fre- 


lrrhe  expression  "new  examination"  is  commonly  used  as  a  collective  designation 
for  tests  or  exercises  which  may  be  objectively  scored  and  which  require  little  writing 
on  the  part  of  those  being  tested. 

2A  test  or  measuring  instrument  is  said  to  be  objective  when  different  persons 
using  it  obtain  the  same  result.  In  other  words,  objective  scoring  eliminates  differences 
due  to  the  fact  that  the  standards  of  different  teachers  and  also  those  of  the  same 
teacher  at  different  times  vary.  In  this  circular  the  expressions  "objective  tests"  and 
"objective  exercises"  will  frequently  be  used  instead  of  "new  examination." 

3The  expression  "traditional"  will  be  applied  to  examinations  of  the  essay  or 
discussion  type  which  have  been  in  common  use  for  many  years. 

4Wood,  Ben  D.  Measurement  in  Higher  Education.  Yonkers:  World  Book  Com- 
pany, 1923.    337  p. 

bRuch,  G.  M.  The  Improvement  of  the  Written  Examination.  Chicago:  Scott, 
Foresman  and  Company,  1924.    193  p. 

"Brinkley,  Sterling  G.  Values  of  New  Types  of  Examinations  in  the  High 
School.  Teachers  College  Contributions  to  Education,  No.  161.  New  York:  Teachers 
College,  Columbia  University,  1924.    121  p. 

7The  chief  of  these  advantages  will  be  presented  later. 

[3] 


quently  when  one  desires  to  test  information,  especially  separate  rather 
than  connected  or  organized  facts.  Objective  exercises  undoubtedly 
secure  some  measure  of  other  achievements  than  mere  knowledge,  such 
as  the  ability  to  connect  cause  and  effect,  to  organize,  to  summarize, 
and  so  forth,  but  the  measures  so  secured  are  not  at  all  adequate. 
Indeed,  in  many  cases,  tests  which  on  the  surface  appear  to  yield 
measures  of  such  abilities  will  be  found  upon  careful  analysis  to  measure 
mere  memory  or  knowledge  of  facts.  For  example,  if  the  question  "Do 
you  think  Grant  was  a  better  general  than  Lee?"  is  asked,  it  is  likely 
that  many  pupils  will  answer  by  giving  a  statement  remembered  from 
textbook  or  teacher.  In  case  a  pupil  does  not  remember  a  satisfactory 
statement,  or  perhaps  even  if  he  does,  he  may  really  form  his  own 
opinion  as  to  the  respective  ability  of  Grant  and  Lee  from  the  pertinent 
facts  he  knows.  Therefore  the  writer  maintains  that  tests  of  the  types 
described  in  this  circular  should  be  used,  for  the  most  part,  when  one 
desires  to  measure  memorized  facts  or  information,  and  that  their  use 
for  this  purpose  should  be  merely  part  of  a  complete  testing  program 
which  also  includes  examinations  of  the  traditional  kind.  The  propor- 
tionate amount  of  testing  to  be  done  by  each  of  the  two  methods  in  a 
particular  situation  depends  chiefly  on  the  kind  of  subject-matter  cov- 
ered and  the  objectives  of  the  course. 

Chief  advantages  of  the  new  examination.  The  new  examination 
has  at  least  three  important  advantages  over  the  traditional  type. 
Probably  the  chief  of  these  is  that  pupil  responses  can  be  scored  upon 
an  objective  or  almost  objective  basis.  This  can  be  accomplished  by 
following  proper  directions  in  administering  and  scoring  the  exercises 
employed.  The  fact  that  objective  or  near-objective  scoring  is  possible 
renders  it  probable  that  reliability8  will  be  high. 

A  second  chief  advantage  is  that  the  amount  of  writing  done  by 
pupils  is  reduced  to  a  minimum.  This  practically  eliminates  the  advan- 
tage of  rapid  and  fluent  writers  over  those  not  so  gifted;  the  fact  that 
they  can  write  more  material  in  the  same  length  of  time  should  not 
result  in  their  receiving  higher  marks  in  most  subjects.  Also  there  is 
much  less  opportunity  for  the  teacher  to  be  influenced,  frequently  un- 
consciously, by  good  or  bad  writing,  spelling,  punctuation,  and  other 
similar  qualities,  which  have  no  connection  with  achievement  except  in 
the  field  of  English. 


8The  reliability  of  a  test  is  the  degree  to  which  a  second  application  yields  scores 
equivalent  to  those  obtained  by  its  first  use.  In  other  words,  it  is  a  measure  of  the 
variable  errors  present  in  the  scores. 

[4] 


The  third  advantage,  which  is  to  some  extent  due  to  the  fact  that 
little  writing  is  required,  is  that  a  much  wider  range  of  achievement 
can  be  measured  or  a  much  larger  sample  can  be  secured  than  is  possi- 
ble by  a  discussion  or  essay  examination  requiring  the  same  amount 
of  time.  This  minimizes  the  likelihood  that  pupils  just  happen  to  know 
or  not  to  know  a  large  enough  proportion  of  the  answers  to  the  ques- 
tions asked  that  their  scores  are  largely  the  result  of  chance.  Other 
minor  advantages  might  be  named  but  it  seems  to  the  writer  that  these 
three  major  ones  are  enough  to  justify  the  use  of  objective  exercises  in 
preference  to  those  of  the  traditional  type  when  one  desires  to  measure 
detailed  factual  knowledge. 

Disputed  points  concerning  the  new  examination.  In  addition  to 
the  advantages  which  this  type  of  examination  is  rather  generally  con- 
ceded to  possess  there  are  certain  other  claims  for  it  concerning  which 
there  are  marked  differences  of  opinion  and,  in  some  cases,  of  experi- 
mental evidence.  One  of  these  is  the  question  of  reliability.  On  the 
whole  the  evidence  presented  on  this  point  indicates  that  the  new  types 
of  exercises  are  somewhat  more  reliable  than  those  of  the  traditional 
type.9 

There  has  also  been  some  argument  concerning  validity.10  One 
writer11  claims  that  the  validity  of  new  examinations  is  somewhat  higher 
than  that  of  the  traditional  ones  but  his  data  do  not  prove  this.  An- 
other12 gives  results  which  indicate  that  there  is  practically  no  difference 
in  the  validity  of  the  two  kinds  except  that  as  a  measure  of  mere  infor- 
mation the  new  type  ranks  slightly  higher.  On  the  whole  one  is 
probably  fairly  safe  in  assuming  that  there  is  no  great  difference  in 
validity  when  compared  with  such  criterion  measures13  as  are  now 
available. 

An  advantage  very  frequently  claimed  for  the  new  examination 
is  that  it  requires  the  expenditure  of  much  less  time  on  the  part  of  the 
teacher.  Wood14  states  that  its  use  is  an  economy  of  time  in  large  classes 
but  not  in  small  ones.  He,  however,  fails  to  state  how  many  pupils  a 
class  must  have  to  be  considered  large.    Brinkley15  states  that  it  takes 


9Wood,  op.  cit.,  p.  188-93. 

Ruch,  op.  cit.,  p.  114. 

"The  validity  of  a  measuring  instrument  is  the  degree  to  which  it  measures  what 
it  purports  to  measure. 

"Wood,  op.  ch.,  p.  188-91  and  elsewhere. 

"Brinkley,  op.  cit.,  p.  58-59. 

"A  criterion  measure  is  one  taken  as  a  basis  for  determining  the  validity  of  some 
other  measure. 

"Wood,  op.  cit.,  p.  198-99. 

"Brinkley,  op.  cit.,  p.  59. 

[5] 


two  and  one-half  times  as  long  to  make  and  score  a  new  examination 
as  a  traditional  one  if  the  two  are  of  equal  length  and  involve  equal 
numbers  of  pupils.  The  numbers  of  pupils  used  in  his  experiments 
were,  however,  fairly  small.  There  appears  to  be  general  agreement 
that  it  requires  more  time  for  the  teacher  to  prepare  the  new  type  than 
the  old.  It  can  readily  be  seen,  therefore,  that  if  only  a  few  pupils  are 
tested  any  economy  in  scoring  will  be  balanced  by  the  added  time 
required  for  making.  On  the  basis  of  his  own  experience  in  using  such 
tests  the  writer  believes  that  for  classes  as  large  as  forty,  and  in  some 
cases  for  those  of  only  twenty-five  or  thirty,  the  total  amount  of  time 
required  of  the  teacher  is  no  greater  for  the  new  than  for  the  old  type 
of  examination.  In  a  few  such  cases  he  has  even  found  it  to  be  less. 
Furthermore,  it  should  not  be  forgotten  that  in  most  cases  a  compara- 
tively unskilled  clerk  can,  if  it  is  desired,  score  exercises  of  the  new 
examination  type,  whereas  this  is  impossible  with  practically  any  tradi- 
tional examination. 

General  principles  of  constructing  and  using  objective  exercises. 
Before  proceeding  to  the  construction  of  the  different  types  of 
objective  tests  it  will  probably  be  well  to  state  a  number  of  general 
principles  which  should  guide  one  in  their  construction  and  use. 

1.  Such  tests  should  be  preceded  by  explicit  directions  as  to  just 
what  is  to  be  done.  If  the  pupils  are  not  familiar  with  the  kind  of 
exercise  to  be  used,  the  directions  should  usually  include  examples. 
Also  the  amount  of  time  to  be  allowed  should  be  stated. 

2.  It  is  important  that  a  fairly  large  number  of  items16  be  included. 
If  tests  are  given  frequently  and  as  a  more  or  less  regular  part  of  the 
procedure  during  class  time  twenty  or  twenty-five  items  may  be  enough. 
Under  other  conditions  fifty  should  probably  be  the  minimum  and  one 
hundred  is  a  still  better  number. 

3.  Care  should  be  exercised  in  the  selection  of  items.  In  some 
cases  practically  all  the  important  items  can  be  included.  In  others,  it 
will  be  necessary  or  desirable  to  make  a  satisfactory  sampling  from  a 
large  number. 

4.  Care  should  be  exercised  in  the  wording  of  the  statements  used 
so  that  ambiguity  will  be  avoided. 

5.  The  exercises  should  be  so  worded  and,  in  case  answers  are 
suggested,  these  should  be  so  selected  that  the  correct  answers  are  not 


18The  term  item  is  used  to  refer  to  the  smallest  unit  or  part  of  a  test  which  calls 
for  a  distinct  answer  or  pupil  response.  Thus  a  single  direct  recall  or  yes-no  question, 
a  single  pair  of  expressions  in  a  matching  exercise,  a  single  word  to  be  denned  or  a  single 
term  to  be  connected  with  the  proper  part  of  a  figure,  is  an  item. 

[6] 


too  evident.  In  other  words,  the  incorrect  answers  should  not  be  so 
absurd  or  far-fetched  that  they  can  be  ruled  out  without  a  satisfactory 
knowledge  of  the  subject-matter  being  tested. 

6.  The  single  items  and  also  the  whole  test  should  be  of  such  a 
degree  of  difficulty  that  no  or  practically  no  perfect  scores  or  zero  scores 
will  be  made. 

7.  The  items  included  should  be  so  arranged  that  there  is  no  reg- 
ular sequence  of  answers.  For  example,  the  positive  and  negative 
statements  or  questions  in  an  alternative  test  should  be  arranged  in 
random  order.  The  same  is  true  of  the  suggested  responses  in  multiple- 
answer  exercises,  of  one  of  the  two  lists  in  matching  exercises,  and 
so  forth. 

8.  The  directions  should  require  the  pupils'  responses  to  be  re- 
corded so  that  scoring  is  as  easy  as  possible. 

9.  The  principle  of  variety  should  be  observed,  that  is,  in  any  one 
course  or  subject  tests  of  a  number  of  different  types  should  be  em- 
ployed. Furthermore,  in  a  single  long  test  it  is  usually  better  to  use 
several  types  of  exercises. 

10.  Pupil's  scores  should  be  tabulated  in  terms  of  points  of  credit 
and  after  this  has  been  done  for  a  whole  class  or  group  these  scores 
should  be  turned  into  marks.17 

Basis  of  choosing  examples.  The  examples  given  in  the  following 
pages  have  been  chosen  to  show  that  the  various  types  may  be  used 
widely  in  different  school  subjects.  In  fact,  almost  any  of  the  kinds 
illustrated  may  be  used  in  any  of  the  commonly  taught  school  subjects. 
An  analysis  of  them,  however,  makes  it  apparent  that  some  are  much 
better  suited  to  use  in  certain  subjects,  others  in  others.  However,  this 
fact  should  not  be  considered  sufficient  cause  for  limiting  very  narrowly 
the  types  used  in  any  one  subject,  and  thus  violating  principle  9. 

Use  of  multigraphed  copies.  It  is  advisable  in  the  use  of  all  of  the 
types  of  exercises  described  in  this  circular  and  necessary  in  the  use 
of  most  of  them  that  mimeographed  or  otherwise  multigraphed  copies 
be  placed  in  the  hands  of  the  pupils.  If  this  is  impossible  or  impracti- 
cable there  are  several  of  the  types  which  teachers   may  employ  by 


17For  a  discussion  of  turning  scores  into  marks,  see  any  of  the  following: 

Monroe,  W.  S.  An  Introduction  to  the  Theory  of  Educational  Measurements. 
Boston:    Houghton  Mifflin  Company,  1923,  p.  292-94. 

Monroe,  W.  S.,  Df.Voss,  J.  C,  and  Kelly,  F.  J.  Educational  Tests  and  Measure- 
ments, Revised  and  Enlarged  Edition.  Boston:  Houghton  Mifflin  Company,  1924, 
p.  425-28. 

Paterson,  Donald  G.  Preparation  and  Use  of  New-Type  Examinations.  Yonkers: 
World  Book  Company,  1925,  p.  70-71. 

[7] 


reading  the  exercises  and  having  the  pupils  record  the  answers  on  blank 
sheets  of  paper.  The  types  with  which  this  is  practicable  are  recall  or 
single  answer,  alternative,  definition,  enumeration,  opposite  and  genus- 
species  varieties  of  association,  and  abbreviation  or  formulae.  The 
writer  does  not  wish  to  be  understood,  however,  as  recommending  that 
these  kinds  of  exercises  be  given  in  this  manner,  unless  it  is  impracti- 
cable to  place  copies  in  the  hands  of  the  pupils.  Of  course  any  of  the 
types  may  be  given  by  writing  the  material  on  the  blackboard,  but  in 
view  of  the  labor  involved  it  is  rarely  desirable  to  do  so. 

Examples  of  objective  exercises.  In  the  following  pages  thirty- 
seven  varieties  of  objective  or  near-objective  exercises  have  been 
grouped  under  fifteen  headings.  Following  the  directions  for  giving  and 
the  examples  of  each  variety  will  be  found  brief  discussions  and  scoring 
directions.  The  varieties  named  and  illustrated  do  not  constitute  an 
absolutely  complete  list,  but  do  include  practically  all  with  which  the 
writer  is  familiar  except  very  minor  variations  or  types  which  are 
unsuited  to  use  in  ordinary  circumstances.  Many  slightly  different 
forms  can  be  made  by  combining  or  mixing  the  features  of  two  or  more 
of  the  kinds  given.  Several  of  the  types  given  illustrate  such  combina- 
tions or  mixtures,  but  many  more  can  be  devised  by  teachers  who  are 
familiar  with  objective  exercises. 

In  selecting  the  examples  which  follow  no  effort  was  made  to 
choose  the  best  possible  ones  in  the  various  subjects.  They  are  not 
based  upon  any  study  of  the  subject-matter  commonly  taught  or  the 
questions  commonly  asked.  In  other  words,  the  examples  in  any  one 
subject,  if  gathered  together,  do  not  constitute  a  test  on  the  minimum 
essentials  of  that  subject.  Practically  all  of  them  are  original  with  the 
writer,  in  so  far  as  their  exact  form  is  concerned.  Many  standardized 
tests,  a  number  of  text-books,  and  several  teachers  were  consulted,  how- 
ever, in  the  search  for  suitable  content. 


[8] 


I.    RECALL  EXERCISES  OR  QUESTIONS  WITH  SINGLE 
CORRECT  ANSWERS 

Directions:18  Answer  each  of  the  following  questions  with  a  single 
word  or  expression.  Write  the  answer  immediately  under  the  first  word 
or  words  of  the  question. 

1.  Who  was  President  of  the  United  States  during  the  war  of  1812-14? 

2.  What  is  the  product  of  x2,  x3  and  x4? 

3.  What  is  the  dative  plural  of  homo? 

4.  What  kind  of  placket  should  be  used  in  a  tailored  woolen  dress? 

This  type,  which  is  one  of  the  easiest  to  prepare  and  which  may 
be  employed  in  all  subjects,  consists  of  questions  having  a  single  word 
or  expression  as  the  only  correct  answer.  Therefore  it  may  be  made 
perfectly  objective,  though  in  practice  this  ideal  is  not  always  attained. 

The  score  is  merely  the  number  of  correct  answers. 


II.    ALTERNATIVE  EXERCISES 

A.   Yes-No  Questions,  Answers  to  be  Underlined 

Directions:  Approximately  half  of  the  following  questions  should  be 
answered  by  "yes"  and  half  by  "no."  If  you  know  or  think  you  know 
which  answer  is  correct,  underline  the  proper  one  of  the  two  words  in 
front  of  the  question.   If  you  do  not  think  you  know,  do  not  guess. 

Yes  No  1.  Did  the  South  favor  the  Mexican  War? 

Yes  No  2.  Did  Browning  write  "The  Idylls  of  the  King?" 

Yes  No  3.  If  3x  +  8=  16  -  x,  does  x  =  2? 

Yes  No  4.  Is  the  ulna  located  in  the  lower  arm? 


18The  directions  for  pupils  and  the  examples  will  be  given  just  as  they  should 
appear  on  the  multigraphed  copies  actually  used. 

[9] 


B.   Yes-No  Questions,  Answers  to  be  Written 

Directions:  Approximately  half  of  the  following  questions  should  be 
answered  by  "yes"  and  half  by  "no."  Write  "yes"  in  front  of  each  ques- 
tion to  which  you  think  it  is  the  answer,  and  "no"  in  front  of  each  to 
which  you  think  it  is  the  answer.  If  you  have  no  idea  which  answer  is 
correct,  do  not  guess. 


1.  Should  corn  immediately  follow  clover  on  the  same  ground? 

2.  Does  "vita"  mean  "error?" 

3.  Is  the  hemlock  an  evergreen? 

4.  Are  invoices  commonly  made  once  a  year? 


C.   True-False  Statements,  Answers  to  be  Underlined 

Directions:  Approximately  half  of  the  following  statements  are  true 
and  half  are  false.  Underline  the  proper  word  before  each  statement 
concerning  whose  truth  or  falsity  you  know  or  think  you  know.  If  you 
do  not  think  you  know,  do  not  guess. 


True  False  1.  The  elm  is  hardier  than  the  maple. 

True  False  2.  Water  is  an  element. 

True  False  3.  A  brush  drawing  should  be  outlined  in  pencil. 

True  False  4.  A  rip  saw  is  used  to  cut  with  the  grain. 


D.   True-False  Statements,  Answers  to  be  Written 

Directions:  Approximately  half  of  the  following  statements  are  true 
and  half  are  false.  Place  a  plus  mark  ( + )  before  each  statement  which 
you  think  is  true,  a  minus  sign  ( — )  before  each  which  you  think  is 
false.   Do  not  guess  unless  you  are  reasonably  sure. 

1.  One  board  foot  contains  1728  cubic  inches. 

2.  Baking  requires  more  heat  than  boiling. 

3.  Most  banks  use  double-entry  bookkeeping. 

4.  Mechanical  drawings  should  usually  be  done  in  pencil  before  be- 

ing inked. 


[10] 


As  the  examples  show,  there  are  several  varieties  of  alternative 
exercises  which  may  be  used.  Questions  to  be  answered  by  either  "yes" 
or  "no"  or  definite  statements  which  are  to  be  marked  either  "true"  or 
"false"  may  be  employed.  In  either  case,  pupil  responses  may  consist  of 
writing  in  the  proper  word  or  underlining  the  proper  one  of  the  two 
words  already  given.  The  actual  question  form  is  better  than  the  state- 
ment form,  since  it  does  not  present  false  statements  which  may  possibly 
interfere  with  learning.  In  preparing  lists  of  questions  or  statements  of 
this  type,  one  should  exercise  care  that  approximately  half  of  them  call 
for  the  one  answer,  half  the  other.  This  type  of  exercise  is  one  of  the 
most  widely  used  and  also  one  of  the  easiest  to  prepare  and  score,  but 
it  has  less  merit  than  a  number  of  the  other  types.  Its  reliability  is 
probably  lower  than  that  of  most  of  the  other  kinds.19  Despite  instruc- 
tions to  the  contrary,  it  is  liable  to  encourage  mere  guessing. 

There  have  been  some  differences  of  opinion  as  to  how  tests  of  this 
type  should  be  scored.  Although  the  method  of  subtracting  wrongs 
from  rights  yields  a  score  little  if  any  more  reliable  than  that  obtained 
by  using  merely  the  number  right,  the  validity  of  the  former  method, 
especially  in  connection  with  instructions  not  to  guess  unless  one  is  fairly 
sure  of  the  answer,  appears  to  be  higher  than  that  of  the  latter  method.20 
Furthermore,  if  those  taking  the  test  know  that  it  is  being  used,  mere 
guessing  will  probably  be  discouraged.  It  also  makes  the  best  theoreti- 
cal correction  for  guessing  and  for  erroneous  answers.  Therefore  the 
writer  recommends  that  on  all  such  tests  the  score  be  the  number  right 
minus  the  number  wrong,  no  account  at  all  being  taken  of  those  omitted. 

III.   INCORRECT  STATEMENTS 
A.   Alternative  Statements  to  be  Corrected 

Directions:  Approximately  half  of  the  following  statements  are  true 
and  half  are  false.  The  latter  have  been  falsified  by  the  insertion  of 
an  incorrect  expression  or  the  use  of  an  incorrect  word  instead  of  a 
correct  one.  Cross  out  the  expression  in  each  false  statement  which 
makes  it  incorrect.  If  the  expression  crossed  out  takes  the  place  of  a 
correct  one,  write  the  correct  expression  immediately  above  the  one 
crossed  out. 


19Ruch,  op.  cit.,  p.  118. 

20Brinkley,  op.  cit.,  p.  61. 

Ruch,  op.  cit.,  p.  114-21,  is  somewhat  favorable  to  taking  merely  the  number 
correct  as  the  score,  but  later,  in  a  paper  delivered  before  the  Society  of  College  Teach- 
ers of  Education  in  February,  1926,  advocates  subtracting  the  number  wrong. 

[11] 


1.  Lead  is  heavier  than  gold. 

2.  Abraham  Lincoln  was  born  in  1812. 

3.  If  x  —  3,  x2  —  2x  +  5  =  2x2  +  x  —  13. 

4.  The  T-square  is  not  needed  in  drawing  a  human  head. 

B.  Incorrect  Statements 
Directions:  Each  of  the  following  statements  is  rendered  incorrect  by 
the  presence  of  a  certain  expression.  In  some  cases  this  is  inserted  in 
an  otherwise  true  sentence,  in  others  it  takes  the  place  of  a  correct  one. 
In  either  case,  the  incorrect  expression  should  be  crossed  out  and  in 
the  second  case  the  correct  one  should  be  written  in  immediately  above 
the  one  crossed  out. 

1.  His  mother  said  he  could  go  to  town. 

2.  Water  freezes  at  0°  Fahrenheit. 

3.  A  United  States  senator  serves  four  years. 

4.  A  peck  contains  2150  cubic  inches. 

The  first  of  the  two  varieties  given  also  involves  the  alternative 
element  and  might  have  been  classified  under  II.  Some  persons  object 
to  the  use  of  this  type  because  of  the  danger  of  erroneous  learning.  The 
writer  believes  that  this  danger  is  present,  but  not  very  great. 

This  type  of  test  should  be  scored  by  counting  the  number  correct. 
It  is  permissible  to  give  half  credit  when  an  incorrect  expression  has 
been  crossed  out,  but  the  correct  one  not  written  in. 

IV.    COMPLETION  EXERCISES 

A.  Simple  Completion  Exercises 
Directions:    The  following  statements  are  to  be  completed  by  writing 
one  and  only  one  word  or  a  number  in  each  blank. 

1.  The is  an  invertebrate. 

2.  " fleur means  "the  red  flower.'' 

3.  Water  boils  at degrees  Fahrenheit. 

4.  The  Congress  of  the  United  States  consists  of members,  of 

whom are   senators   and members   of  the  lower 

house.    Senators  are  elected  for years  and  representa- 
tives for years.  The presides 

over  the  senate,  whereas  a is  elected 

to  preside  in  the  house  of  representatives. 

[12] 


B.   Completion  Exercises  with  Suggested  Answers 
Directions:    Complete  the  following  statements  by  writing  one  of  the 
words  found  in  the  list  at  the  right  of  the  page  in  each  blank.    Do  not 
use  the  same  word  twice. 

Adams 

1 won  the  battle  of  New  Orleans.  Clay 

2 was  first  Secretary  of  the  Treasury.     Hamilton 

3 was  a  leader  for  Nullification.  Haynes 

4.  was  President  during  1797-1801.  Jackson 

The  completion  type  of  test  is  one  of  the  most  commonly  used  of 
the  newer  objective  forms.  From  the  standpoint  of  pupil  response  it 
is  probably  one  of  the  most  valuable  tests  of  this  kind.  It  is  very 
difficult,  however,  to  construct  a  simple  completion  test  so  that  the 
scoring  is  highly  objective.  Pupils  are  almost  certain  to  give  answers 
which  it  is  difficult  to  evaluate  as  either  absolutely  right  or  wrong.  This 
difficulty  can  be  eliminated  by  requiring  answers  to  be  chosen  from  a 
given  list.  Although  this  type  generally  consists  of  a  number  of  sen- 
tences with  one  or  more  blanks  in  each,  connected  paragraphs  contain- 
ing a  number  of  blanks  may  be  used  instead. 

Completion  tests  are  scored  by  counting  the  number  of  statements 
correctly  completed  or  the  number  of  blanks  correctly  filled  in.  Some- 
times half  credit  is  allowed  for  answers  which  are  not  entirely  correct 
and  yet  cannot  be  classified  as  absolutely  incorrect. 

V.    MULTIPLE-ANSWER  OR  RECOGNITION  EXERCISES 

A.   Multiple-Answer  Exercises  with  only  one  Correct  Answer 

Directions:  Each  of  the  following  statements  can  be  correctly  com- 
pleted by  the  use  of  one  and  only  one  of  the  expressions  contained 
within  the  parenthesis.  Underline  the  correct  expression  in  each 
sentence. 


9h     h 

1.  A  side  of  an  equilateral  triangle  equals  (J£\/fr  h\/5?^ — 7='  _a/3)- 

2.  The  President  must  be  (21,  25,  30,  35,  40)  or  more  years  of  age. 

3.  Discounts  are  (never,  rarely,  sometimes,  usually)  in  percents. 

4.  If — represents  the  earth's  distance  from  the  sun,  that  of  Jupiter  from 
the  sun  is  best  represented  by  ( , , , ). 

[13] 


B.   Multiple-Answer  Exercises  with  Answers  of  Varying 
Degrees  of  Merit 

Directions:    Underline  the  one  expression  in  the  parenthesis  of  each 
sentence  which  makes  the  best  statement. 


1.  (Copper,  iron,  paper,  rubber)  is  a  good  conductor. 

2.  The  (pine,  oak,  elm,  pear,  ash)  is  a  desirable  shade  tree. 

3.  Twelve  year  old  children  should  have  (8,  9,  10,  11,  12)  hours  sleep. 

4.  (Very  few,  few,  some,  most)  final  consonants  are  sounded  in  French. 


C.  Multiple-Answer  Exercises  with  one  or  more  Correct  Answers 

Directions:    Underline  all  of  the  expressions  in  each  parenthesis  which 
make  true  statements  when  taken  with  the  rest  of  the  sentence. 


1.  (Poe,  Longfellow,  Scott,  Macaulay,  Irving)  were  Americans. 

2.  Louisiana  raises  much  (corn,  oats,  rice,  cotton,  sugar-cane). 

3.  A  (chisel,  nail-set,  lathe,  plane)  is  needed  in  making  a  drawer. 

4.  (Alto,  campo,  magro,  polmo,  lado)  are  adjectives. 


D.   Compound  Multiple-Answer  Exercises 

Directions:    Underline  the  one  expression  in  each  line  which  is  closely 
associated  with  the  expression  given  in  the  first  line. 


1.  Raleigh. 

Virginia,   North    Carolina,    South   Carolina,   Alabama,   Georgia. 
(Population)   15,000,  25,000,  40,000,  50,000,  75,000. 

2.  Henry  Clay. 

"Tennessee,  Kentucky,  Virginia,  Ohio. 

Whig,  Democrat,  Federalist,  Republican. 

Governor,  Vice-President,  Attorney-General,  Congressman. 

E.  Multiple-Description  Exercises 

1.  Directions:   Place  a  check  mark  (V )  in  front  of  the  paragraph  which 
best  describes  the  city  of  Pittsburgh. 

[14] 


a.  Pittsburgh  is  a  city  of  about  1,000,000  population,  located  in  the 
western  part  of  Pennsylvania  on  the  Ohio  river.  Situated  in  the  center 
of  a  prosperous  agricultural  region,  it  is  a  leading  center  in  the  manu- 
facture of  flour  and  dairy  products. 

b.  The  city  of  Pittsburgh,  with  a  population  of  more  than  half-a-million, 
is  located  at  the  junction  of  the  Allegheny  and  Monongehela  rivers.  Be- 
cause of  its  proximity  to  coal  fields  and  its  transportation  facilities,  it 
has  become  one  of  the  centers  of  the  iron  and  steel  industry. 

c.  Pittsburgh,  the  second  largest  city  of  Pennsylvania,  is  one  of  the 
leading  live-stock  markets  of  this  country.  It  is  also  noted  for  its  iron 
and  steel  manufactures.    It  is  often  called  the  "Smoky  City." 


2.  Directions:  Place  a  check  mark  (V)  in  front  of  the  statement  which 
best  describes  the  human  heart. 

a.  The  heart  is  a  sort  of  pump,  located  in  the  upper  left  part  of  the 
chest.  It  is  about  the  size  of  a  quart  measure,  but  conical,  the  larger 
end  being  up.  It  consists  of  two  auricles,  on  the  right,  and  two  ven- 
tricles, on  the  left,  with  valves  between. 

b.  The  heart  is  the  center  of  the  circulatory  system  and  is  located  within 
the  lungs.  It  resembles  a  large  pear  in  size  and  shape.  The  two  ventri- 
cles receive  the  blood  from  the  veins  and  pass  it  on  to  the  auricles 
which  force  it  into  the  arteries. 

c.  The  human  heart  is  essentially  a  group  of  muscles  which  operates  as 
a  pump  to  cause  the  blood  to  circulate.  It  consists  of  four  chambers,  or 
tanks,  the  right  and  left  auricles  above,  and  the  right  and  left  ventricles 
below,  with  valves  to  prevent  the  blood  from  flowing  in  the  wrong  di- 
rection.   It  is  about  the  size  of  one's  fist. 

d.  The  heart  is  a  double  pump  which  receives  the  blood  from  the  body, 
purifies  it,  and  sends  it  back  again.  The  work  is  accomplished  by  an 
elaborate  system  of  reservoirs,  valves  and  filters.  The  lungs  supply 
the  energy,  or  fuel,  necessary  to  the  heart's  work. 

Everything  considered,  multiple-answer  exercises  are  among  the 
best  of  those  mentioned  in  this  circular.  If  one  is  careful  in  selecting 
the  suggested  answers,  scoring  can  be  made  absolutely  objective.  Fur- 
thermore, in  many  cases  one  can  make  the  degree  of  discrimination 
which  must  be  exercised  by  pupils  as  fine  as  is  desired.    It  is  probably 

[15] 


most  common  to  suggest  either  four  or  five  answers  though  it  is  not 
unusual  to  see  two,  three,  six  or  seven  and  occasionally  even  more  than 
seven,  used.  Usually  each  answer  is  a  single  expression  but  sometimes 
whole  sentences  or  even  paragraphs  are  used,  as  in  E. 

The  methods  of  scoring  the  various  types  of  multiple-answer  ex- 
ercises differ.  Type  A  is  ordinarily  scored  by  counting  merely  the 
number  of  correct  responses.  Sometimes,  however,  because  of  the  pos- 
sibility of  guessing  correctly,  a  correction  is  made  by  using  the  fol- 
lowing formula:     Score  =  R  — In  this  R  —  the  number  of 

N  -  1 
right  answers,  W  the  number  of  wrong  ones  and  N  the  number  of 
suggested  answers  in  each  exercise.  Thus  if  three  possible  answers  are 
given  one-half  of  the  number  wrong  is  subtracted  from  the  number 
right,  if  four  possible  numbers,  one-third  the  number  wrong  is  sub- 
tracted, if  five  possible  answers,  one-fourth,  and  so  on.  When  only  three 
or  four  answers  are  given,  it  is  probably  wise  to  use  this  formula,  but  for 
more  than  four  the  increase  in  the  validity  of  the  scores  is  scarcely 
great  enough  to  justify  the  required  computations.21 

The  scoring  of  type  B  is  somewhat  more  complicated  than  that  of 
the  previous  one.  Each  exercise  should  contain  one  best  answer  which 
is  valued  at,  say,  five  points,  at  least  one  absolutely  incorrect  answer 
for  which  no  credit  is  allowed,  and  one  or  more  answers  worth  varying 
amounts  between  the  best  and  the  worst.  A  pupil's  score  consists  of  the 
total  number  of  points  of  credit  given  for  the  expressions  he  underlines. 

The  score  for  type  C  is  usually  the  number  of  correct  words  under- 
lined. A  correction  may  be  applied  by  subtracting  the  number  of  in- 
correct ones  underlined,  however. 

Type  D,  compound  multiple-answer  exercises,  may  be  scored  in 
either  one  of  two  ways.  An  exercise  may  be  counted  correct  only  if  the 
proper  expression  in  each  line  is  indicated  or  partial  credit  may  be 
given  if  the  correct  expressions  are  underlined  in  some  lines  but  not  in 

W 

all.    In  the  latter  case  the  formula,  Score  =  R  •—   ,  may  be  em- 


ployed. 

Variety  E  may  be  scored  like  either  A  or  B. 


N  -  1 


1Ruch,  op.  cit.,  p.  114-21. 

[16] 


VI.    MULTIPLE-REASON  EXERCISES 

A.   Multiple-Reason  Exercises  with  only  one  Correct  Reason 

Directions:  One  and  only  one  of  the  reasons  given  after  each  state- 
ment is  correct.  Indicate  the  correct  one  by  making  a  check  mark  (V) 
just  before  it. 


1.  We  know  that  animal  life  has  existed  millions  of  years  because 

a.  Existing  forms  could  not  have  developed  in  less  time. 

b.  Human  records  from  early  times  state  it. 

c.  It  is  proven  by  fossils  which  have  been  discovered. 

d.  We  know  that  proper  food  for  animals  existed. 

2.  Caesar  conquered  Gaul  because 

a.  The  Gauls  were  poor  fighters. 

b.  His  army  was  better  armed,  disciplined  and  led. 

c.  The  Romans  were  larger  and  stronger  than  the  Gauls. 

d.  He  had  a  larger  army  than  his  foes. 


B.   Multiple-Reason  Exercises  with  Reasons  of  Varying  Degrees 
of  Merit 

Directions:    Place  a  check  mark  (V)  in  front  of  the  best  reason  for 
each  of  the  statements  given  below. 


1.  The  North  won  the  Civil  War  because 

a.  It  had  better  generals. 

b.  It  had  greater  resources  in  men,  money  and  materials. 

c.  Its  soldiers  were  braver  and  hardier. 

d.  Its  navy  blockaded  southern  ports. 

e.  Lincoln  was  a  better  statesman  than  Davis. 

2.  Coal  is  a  better  fuel  than  wood  because 

a.  It  burns  longer. 

b.  It  yields  more  heat. 

c.  It  is  easier  to  procure. 

d.  It  costs  less. 


[17] 


C.   Multiple-Reason  Exercises  with  one  or  more  Correct  Reasons 
Directions:    Place  a  check  mark  (V)  in  front  of  each  of  the  reasons 
given  below  which   apply  to  the   statement  under  which  it  is  found. 
There  may  be  only  one  correct  reason  for  each  statement  or  there  may 
be  more  than  one. 

1.  Great  Britain  developed  more  commerce  than  France  because 

a.  It  was  more  necessary  to  her  prosperity. 

b.  France  did  not  attempt  to  stimulate  commerce. 

c.  France  had  little  access  to  the  sea. 

d.  Great  Britain  acquired  more  colonies. 

2.  Literature  developed  slowly  in  America  during  1700-1800  because 

a.  Its  development  requires  leisure. 

b.  The  intellectual  level  here  was  low. 

c.  European  writers  were  imitated  too  much. 

d.  The  material  demands  of  daily  life  were  heavy. 

e.  Books  could  not  be  printed  here. 

This  kind  of  test  is  very  similar  to  the  multiple-answer  type,  the 
difference  being  that  instead  of  statements  to  be  completed  by  the  se- 
lection of  the  proper  expressions  it  contains  statements  followed  by  sev- 
eral possible  reasons  of  which  the  correct  ones  are  to  be  indicated.  The 
three  varieties  of  this  type  are  similar  to  the  first  three  varieties  given 
under  the  multiple-answer  type. 

The  three  varieties  of  multiple-reason  tests  just  given  are  scored 
in  the  same  method  as  the  corresponding  varieties  of  multiple-answer 
tests. 

VII.    MATCHING  EXERCISES 

Directions:  Each  word  in  the  list  to  the  right  is  the  English  translation 
of  one  of  the  Latin  words  in  the  list  to  the  left.  Place  the  letter  pre- 
ceding each  English  word  before  the  Latin  word  of  which  it  is  the  trans- 
lation. 


aequor 

a. 

shoulder 

ora 

b. 

shore 

umerus 

c. 

couch 

clipeus 

d. 

shield 

cubile 

e. 

sea 

advena 

f. 

stranger 

[18] 


Directions:  Each  of  the  phrases  in  the  right-hand  column  is  descriptive 
of  one  of  the  men  named  in  the  left-hand  column.  Make  the  proper 
connections  by  writing  the  number  preceding  the  phrase  in  front  of  the 
name  of  the  man  with  whom  it  should  be  associated. 

Tennyson  1.  Modern  American  novelist. 

Kipling  2.  "The  Quaker  Poet." 

Whittier  4.  Modern  English  writer  of  eastern  tales. 

Wright  4.  American  poet,  wrote  much  for  children. 

Riley  5.  Poet  laureate,  author  of  "In  Memoriam." 

In  the  use  of  this  type  one  should  take  care  not  to  make  the  cor- 
responding lists  contain  too  many  items,  since  so  doing  results  in  con- 
siderable waste  of  time  in  looking  up  and  down  the  list  to  find  the  de- 
sired expressions.  From  ten  to  twenty  items  is  probably  the  optimum 
number  for  a  single  list.  If  it  is  desired  to  give  a  longer  test  than  this 
the  material  should  be  grouped  into  parts.  An  objection  sometimes 
made  to  this  type  of  test  is  that  if  a  pupil  knows  and  has  marked  most 
of  the  items  it  is  easy  to  mark  the  few  remaining  correct  ones  by  a  pro- 
cess of  elimination  and  careful  guessing.  The  writer  does  not  believe 
that  this  objection  is  very  serious.  It  can  be  eliminated,  however,  by 
placing  more  items  in  one  list  than  in  the  other. 

The  score  is  the  number  correct. 

VIII.    DEFINITIONS 

A.   Simple  Definitions  or  Explanations 
Directions:    Define  or  explain  each  of  the  following  words  as  concisely 
and  clearly  as  possible. 

1.  a.  protozoa  2.  a.  siderite 

b.  felidae  b.  compound 

c.  eugenics  c.  osmosis 

d.  embryo  d.  ion 

B.  Same  or  Opposites 
Directions:  The  two  expressions  in  each  of  the  following  pairs  have 
either  practically  the  same  meaning  or  directly  opposite  meanings.  In- 
dicate those  which  mean  the  same  by  writing  an  "s"  in  front  of  the 
pair  and  those  which  have  opposite  meanings  by  writing  an  "o"  in  front 
of  the  pair. 

[19] 


1.  a.  treaty,  protocol  2.  a.  bon,  mauvals 

b.  plenary,  restricted  b.  aller,  marcher 

c.  repeal,  abrogate  c.  cesser,  continuer 

d.  executive,  administrative  d.  venir,  arriver 

C.    Distinguishing  Exercises 
Directions:    Indicate  clearly  the  difference  or  distinction  between  each 
of  the  following  pairs  of  words.   Do  so  in  as  few  words  as  possible. 
1.  a.  vapor,  steam  2.  a.  fry,  broil 

b.  heat,  light  b.  boil,  stew 

c.  sound,  hearing  c.  protein,  fat 

d.  static,  current  d.  bake,  roast 

These  kinds  of  exercises  are  not  new,  nor  are  A  and  C  highly  objec- 
tive. They  are  included  here,  however,  because  they  do  offer  the 
opportunity  of  marking  on  a  definite  point  system.  It  is  almost  always 
possible  to  grade  a  definition  either  as  right  or  wrong,  or  as  right,  half 
right  or  wrong.  The  second  and  third  varieties  given  do  not  call  for 
definitions  but  probably  belong  under  this  general  heading. 

The  score  on  types  A  and  C  may  either  be  taken  as  merely  the 
number  right  or  two  points  may  be  given  for  each  entirely  correct  and- 
one  for  each  partially  correct  definition.  For  type  B  the  score  is  the 
number  right  minus  the  number  wrong. 

IX.    ENUMERATION 
A.    Complete  Enumeration 

1.  Name  the  presidents  from  Lincoln  to  Coolidge. 

2.  Give  the  nominatives  of  all  the  personal  pronouns. 

3.  Name  the  different  varieties  of  maples. 

4.  List  all  the  factors  of  a4  —  a2b2. 

B.   Partial  Enumeration  or  Giving  Examples 

1.  Name  five  foods  rich  in  starch. 

2.  Mention  four  breeds  of  dairy  cattle. 

3.  Name  six  confederate  generals. 

4.  List  five  common  coal-tar  products. 

This  also  is  not  a  new  type  of  test  exercise  but  one  which  has  been 
in  fairly  common  use.    With  a  little  care  it  can  be  made  entirely  ob- 


[20] 


jective.  Furthermore,  it  has  one  of  the  widest  ranges  of  usefulness  of 
any  of  the  objective  tests. 

In  both  varieties  of  enumeration  one  point  credit  should  be  allowed 
for  each  item  correctly  given.  Thus  the  total  number  of  points  of 
credit  on  each  exercise  should  equal  the  number  of  items  to  be  named 
in  that  exercise. 

X.    ASSOCIATION  EXERCISES 
A.  Opposites 

Directions:  Write  the  opposite  of  each  of  the  following  words  imme- 
diately after  it. 

1.  a.  senor  2.  a.  sharp 

b.  bianco  b.  pianissimo 

c.  bien  c.  crescendo 

d.  corto  d.  allegro 

B.    Genus-Species 

1.  To  what  family  does  each  belong? 

a.  cat 

b.  wolf 

c.  whale 

d.  earthworm 

e.  eagle 

2.  Name  a  leading  operatic  composer  of  each  country. 

a.  France 

b.  Prussia 

c.  Hanover 

d.  Bohemia 

C.   Connected  Terms 

Directions:  Underline  all  words  in  each  parenthesis  which  are  in  some 
way  closely  connected  with  or  related  to  the  word  preceding  the  paren- 
thesis. 

1.  Noun  (case,  conjugation,  number,  voice,  subject) 

2.  Italy  (Mazzini,  Thaddeus,  Cavour,  Kossuth,  Garibaldi) 

3.  Cattle  (Hereford,  Jersey,  Poland  China,  Ayrshire,  Arab) 

4.  Dress  slip  (satin,  wool,  silk,  sateen,  jersey) 

[21] 


D.    Disconnected  Terms 

Directions:    Cross  out  the  one  word  in  each  parenthesis  not  closely  con- 
nected with  or  related  to  the  word  before  the  parenthesis. 


1.  Electricity  (fermentation,  ohm,  potential,  volt,  static) 

2.  Triangle  (vertex,  right,  equilateral,  radius) 

3.  Egg  (fat,  sugar,  protein,  calcium,  iron,  phosphorus) 

4.  Future  (hablare,  vivira,  comia,  estudiaran,  sentiras) 


E.   Disconnected  Terms 

Directions:    Cross   out  all  words   in  each   parethensis   which   are  not 
closely  connected  with  or  related  to  the  word  before  the  parenthesis. 


1.  Compound  (hydrogen,  air,  water,  oxygen,  carbon,  salt) 

2.  Georgia  (Atlanta,  Columbus,  Jackson,  Tampa,  Savannah) 

3.  Arm  (femur,  tibur,  ulna,  radius,  carpal,  phalanges) 

4.  Wool  (dimity,  broadcloth,  calico,  serge,  tweed) 


These  exercises  involve  the  association  of  two  or  more  words  in 
any  one  of  several  ways.  The  first  type  included  might  have  been 
classified  under  definitions  but  it  seemed  to  the  writer  slightly  better 
to  place  it  here.  Unless  considerable  care  is  exercised  in  selecting  the 
items  to  be  included  in  type  A,  scoring  will  not  be  perfectly  objective, 
since  it  will  be  possible  for  pupils  to  give  words  which  are  not  entirely 
opposite  to  the  ones  given  and  yet  come  fairly  near  to  being  so.  It  will 
be  noted  that  variety  B  is  in  no  sense  new. 

For  types  A  and  B  the  score  is  the  number  right. 

The  usual  method  of  scoring  C  and  E  is  to  count  the  number  of 
words  correctly  underlined.  Sometimes,  however,  the  number  incor- 
rectly underlined  is  subtracted  from  the  former  number.  It  is  probably 
better  to  make  this  correction. 

Type  D  is  scored  like  the  multiple-answer  type  with  only  one  cor- 
rect answer. 

[22] 


XI.   ARRANGE-IN-ORDER  EXERCISES 

1.  Arrange  in  chronological  order,  beginning  with  the  earliest, 
a.  Xerxes       b.  Cyrus       c.  Philip       d.  Pericles       e.  Ptolemy 

2.  Arrange  in  order  of  protein  content,  from  least  to  most, 
a.  oatmeal      b.  round  steak      c.  milk      d.  white  bread. 

3.  Arrange  in  order  of  area,  from  smallest  to  largest, 

a.  Siberia      b.  China      c.  Japan      d.  India       e.  Persia. 

4.  Arrange  in  order  of  specific  gravity,  from  lowest  to  highest. 
a.  lead      b.  iron      c.  aluminum      d.  gold      e.  copper. 


This  is  one  of  the  types  which  may  be  made  entirely  objective. 
There  are,  however,  several  possible  methods  of  scoring.  The  simplest 
way  is  merely  to  give  one  point  credit  for  each  term  placed  where  it 
should  be  regardless  of  where  the  other  terms  are  placed.  A  more  accu- 
rate method  but  one  which  increases  the  labor  of  scoring  considerably  is 
as  follows:  Let  each  item  count  for  one  point  less  credit  than  the  number 
of  items.  In  other  words,  if  there  are  six  items  let  each  count  five  points. 
Deduct  the  difference  in  rank  between  where  each  item  should  be  and 
where  it  has  been  placed  from  the  number  of  points  allowed  for  each 
item.  For  example,  if  an  item  should  be  second  and  has  been  placed 
fifth,  three  points  should  be  deducted  from  the  number  of  points  allowed 
for  each  item.  The  score  is  then  the  sum  of  the  number  of  points  left 
after  these  deductions  have  been  made. 

XII.    IDENTIFICATION   EXERCISES 
A.   Picture  or  Diagram  Numbered  or  Lettered 
1.  Directions:    Each  of  the  numbers  on  the  triangle  indicates  the  loca- 
tion of  one  of  the  parts  named  in  the  list  of  terms  at  the  right.    Make 
the  proper  connections  by  writing  each  number  in  front  of  the  appro- 
priate term. 


right  angle 

external  angle 

acute  angle 

hypotenuse 

vertex 

leg 


[23] 


2.  Directions:  Place  each  of  the  letters  found  on  the  figure  in  front  of 
the  name  of  the  bone  upon  or  nearest  to  which  it  is  found,  if  this  name 
appears  in  the  accompanying  list. 


radius 

humerus 

femur 

ulna 

tibia 

carpals 

tarsals 


B.   Picture  or  Diagram  to  be  Numbered  or  Lettered 
1.  Directions:    Place  the  letter  found  in  front  of  each  term  at  the  right 
on  the  part  of  the  figure  to  which  the  term  refers. 


a.  spur 

b.  comb 

c.  hock 

d.  ear-lobe 

e.  wing-covert 

f.  tail-covert 


[24] 


2.  Directions:   Place  the  number  found  in  front  of  each  Canadian  city 
in  the  list  at  the  right  on  the  map  at  the  spot  where  the  city  is  located. 

1.  Denver 

2.  Toronto 

3.  Sydney 

4.  Quebec 

5.  Halifax 

6.  Manchester 

7.  Ottawa 

8.  Vancouver 

9.  Winnipeg 
10.  Montreal 


This  type  is  one  which  it  is  practically  impossible  to  use  in  several 
subjects.  It  involves  the  use  of  a  picture,  figure  or  diagram  and  the 
identification  of  various  parts  thereof  as  is  shown  by  the  examples. 
The  identification  may  be  made  by  having  the  picture  or  figure  num- 
bered or  lettered  and  connecting  the  numbers  or  letters  with  the  proper 
terms  or  the  terms  may  be  numbered  or  lettered  and  the  symbols 
placed  in  the  proper  place  on  the  picture  or  diagram.  This  type  may 
also  be  varied  by  giving  a  list  which  contains  fewer,  just  as  many,  or 
more  terms  than  the  picture  or  diagram  appears  to  call  for.  Exercise  2 
under  A  combines  the  first  and  last  ideas  by  omitting  from  the  list  the 
name  of  one  of  the  lettered  parts  and  at  the  same  time  including  several 
terms  not  connected  with  the  figure. 

Type  B  is  not  entirely  objective  since  it  is  frequently  difficult  to 
determine  whether  the  letter  or  number  has  been  placed  on  exactly  the 
right  part  of  the  picture  or  not. 

The  score  on  identification  exercises  is  the  number  correct. 


XIII.   ABBREVIATIONS  OR  FORMULAE 

A.  Abbreviations,  Symbols,  or  Formulae  to  be  Given 

Directions:  Give  the  chemical  symbols  for  each  of  the  following 

a.  hydrogen  chloride 

b.  copper  sulphate 

c.  carbon  dioxide 

d.  sodium 


[25] 


2.  Directions:    Give  the  formulae  for  each  of  the  following: 

a.  area  of  circle 

b.  volume  of  sphere 

c.  area  of  hexagon 

d.  surface  of  sphere 

B.   Abbreviations,  Symbols,  or  Formulae  to  be  Expanded 

1.  Directions:    Give  the  word  or  words  for  which  each  of  the  following 
stands: 

a.  C.O.D. 

b.  F.  O.  B. 
c  @ 

d.  " 

2.  Directions:    Give  the  musical  term  for  which  each  of  the  following 
stands: 

a.  ff 

b.  ppp 

c.  M.  D. 

d.  m.  f. 

This  is  another  kind  of  objective  test  which  is  not  at  all  new.  It  has 
little  place  in  some  subjects  but  in  others  it  has  a  wide  range  of  use. 
The  scoring  may  be  made  entirely  objective. 

The  score  is  the  number  correct. 

XIV.    CLASSIFICATION 
A.   Exercises  with  one  Extraneous  Word 
Directions:    In  each  line  there  is  one  word  which  is  unlike  or  of  a  dif- 
ferent sort  from  the  rest.   Cross  it  out. 

1.  radish,  salsify,  onion,  beet,  parsnip. 

2.  noun,  verb,  pronoun,  phrase,  conjunction. 

3.  hasta,  tempora,  porta,  silva. 

4.  circuit,  supreme,  district,  township. 

B.   Exercises  with  one  or  more  Extraneous  Words 
Directions:    Cross  out  all  words  in  each  line  which  are  unlike  or  of  a 
different  class  from  the  majority  of  the  words  in  that  line. 

1.  Grant,  Jackson,  Lee,  Sheridan,  Johnston,  Early. 

2.  Cedric,  Bois-Gilbert,  Saladin,  Rowena,  Bertha,  Rebecca. 

3.  calico,  gingham,  flannel,  serge,  madras,  dimity. 

4.  oak,  fir,  walnut,  mahogany,  gum,  pine. 


[26] 


This,  and  also  the  following  type,  seems  to  be  among  the  least 
valuable  for  the  purpose  of  testing  achievement,  since  exercises  of  this 
kind  are  probably  in  too  large  a  degree  measures  of  general  intelligence. 
They  do,  however,  yield  some  measure  of  achievement  and  may  receive 
a  limited  use  for  this  purpose. 

The  scoring  of  these  two  varieties  is  the  same  as  that  of  varieties 
D  and  E  of  the  association  type. 

XV.    ANALOGIES 

A.  Analogies  with  no  Answer  Suggested 
Directions:    Write  on  the  blank  line  the  proper  word  or  expression  to 
complete  each  of  the  following  statements: 

1.  a2  is  to  a6  as  x3  is  to 

2 is  to  ceder  as  hablaron  is  to  hablar. 

3.  Lain  is  to  lie  as  is  to  sit. 

4.  Sharp  is  to  flat  as  crescendo  is  to 

B.  Analogies  with  Suggested  Answers 
Directions:    Underline  the  one  of  the  answers  in  the  parenthesis  which 
makes  the  truest  statement. 

1.  Square  is  to  cube  as  circle  is  to  (radius,  solid,  sphere,  circumference). 

2.  Gallons  per  minute  are  to  pressure  as  watt  hours  are  to  (volts,  ohms, 
amperes,  mhos). 

3.  Red  is  to  green  as  (orange,  brown,  violet,  yellow)  is  to  blue. 

4.  Je  is  to  moi  as  ils  is  to  (il,  elles,  leur,  eux). 

This  type  of  exercise  is  much  used  in  intelligence  tests  and  usually 
belongs  there  rather  than  in  an  achievement  test  even  though  it  deals 
with  subject-matter  commonly  taught.  On  the  other  hand,  there  are 
probably  some  cases  in  which  such  a  test  of  subject-matter  may  be 
used  with  good  results.  At  least  two  varieties  of  this  type  may  be  used, 
one  of  which  is  essentially  of  the  single  correct  answer  or  completion 
type,  the  other  of  the  multiple-answer  type. 

The  score  on  type  A  is  the  number  of  correct  answers,  whereas  B 
is  scored  in  the  same  manner  as  the  multiple-answer  type  with  only 
one  correct  answer. 

[27] 


CIRCULARS  OF  THE  BUREAU  OF  EDUCATIONAL  RESEARCH,  COLLEGE 
OF  EDUCATION,  UNIVERSITY  OF  ILLINOIS,  URBANA,  ILLINOIS 

No.  19.  Streitz,  Ruth.    Provisions  for  Exceptional  Children  in  191  Illinois  Cities. 

No.  20.  McClusky,  Frederick  Dean.    Place  of  Moving  Pictures  in  Visual  Education. 

No.  21.  Monroe,  Walter  S.    Announcement  of  the  Bureau  of  Educational  Research 
for  1923-24. 

No.  22.  Odell,  Charles  W.   Provisions  for  the  Individual  Differences  of  High  School 
Pupils. 

No.  23.  Monroe,  Walter  S.    Educational  Guidance  in  High  Schools. 

No.  24.  Nolan,  Aretas  W.  The  Project  in  Education  with  Special  Reference  to  Teach- 
ing Agriculture. 

No.  25.  Monroe,  Walter  S.  and  Clark,  John  A.    Measuring  Teaching  Efficiency. 

No.  26.  Barton,  H.  J.,  Clark,  E.  L.,  Pence,  Helen,  and  others.  Notes  on  the  Teaching 
of  Latin  in  High  Schools. 

No.  27.  Streitz,  Ruth.  Educational  Diagnosis. 

No.  28.  Staley,  Seward  C.    The  Program  of  Sportsmanship  Education. 

No.  29.  Odell,  Charles  W.  The  Use  of  the  Question  in  Classroom  Instruction. 

No.  30.  Odell,  Charles  W.    The  Evaluation   and  Improvement  of  School  Buildings, 
Grounds  and  Equipment. 

No.  31.  Monroe,  Walter  S.   The  Planning  of  Teaching. 

No.  32.  Miller,    F.    J.,    Flickinger,    R.    C,    Sargent,    Rachel    L.,    Luke,    Ethel    J., 
Thompson,  Glenna  D.,  and  others.   Latin  in  High  Schools. 

No.  33.  Odell,    Charles    W.     Educational    Tests    for    Use    in    Elementary    Schools, 
Revised. 

No.  34.  Odell,  Charles  W.    Educational  Tests  for  Use  in  High  Schools,  Revised. 

No.  35.  Monroe,  Walter  S.    The  Making  of  a  Course  of  Study. 

No.  36.  Reagan,    George    W.     Principles    Relating    to    the    Engendering   of    Specific 
Habits. 

No.  37.  Herriott,  M.  E.    How  to  Make  a  Course  of  Study  in  Arithmetic. 

No.  38.  Odell,  Charles  W.    The  Assignment  of  Lessons. 

No.  39.  Prescott,    Henry    W.,    Flickinger,  Roy    C,    Woodruff,    Laura    B.,    Whaley, 
Irene  G.,  and  others.    Appreciation  of  Latin. 

No.  40.  Orata,  Pedro  T.   Adaptation  of  Subject-Matter  and  Instruction  to  Individual 
Differences  in  the  Elementary  School. 

No.  41.  Herriott,  M.  E.    Modifying  Technique  of  Instruction  for  Gifted  Children. 

No.  42.  Herriott,  M.  E.    How  to  Make  a  Course  of  Study  in  Reading. 

No.  43.  Monroe,  Walter  S.    Projects  and  the  Project  Method. 

Xo.  44.  Odell,  Charles  W.    Objective  Measurement  of  Information. 

A  limited  number  of  copies  of  these  educational  circulars  are  available  for  free  distribution  to 
superintendents  and  teachers  in  Illinois.  We  shall  be  glad  to  add  to  our  mailing  list  for  these 
circulars  the  names  of  any  teachers  or  superintendents  who  care  to  receive  them  regularly.  We 
shall  be  glad  also  to  send  additional  copies  of  any  circular  to  superintendents  or  principals  for 
distribution  among  their  teachers.  Address  all  communications  to  the  Bureau  of  Educational 
Research,    University   of   Illinois. 


